iT邦幫忙

2021 iThome 鐵人賽
DAY 27
0
Software Development

Redis還在學系列 第 27 篇

Day27 Redis架構實戰-Redis叢集Slot分流機制

13th鐵人賽 redis
2537 瀏覽

  • 分享至 

  • xImage
    •  

Redis叢集Slot分流機制

# Redis叢集配置
                          client
                            |
                            V
--------------------------------------------------------------
       Redis               Redis               Redis
     (Master)            (Master)            (Master)
    |        |           |       |           |       |
    V        V           V       V           V       V
  Redis    Redis       Redis    Redis       Redis    Redis
(Replica) (Replica)  (Replica) (Replica)  (Replica) (Replica)
--------------------------------------------------------------
示意   1/3 Data            1/3 Data             1/3 Data
  • 資料Slot分流機制
    • Redis Server叢集將Key Space分成16384個slots，這些slots可以分散到最多16384個主節點上．
    • 由於Redis Server叢集採用Gossip協議實施無中心式叢集，如果超過1000個主節點時因為Gossip協議需要頻繁的訊息傳輸所以效率會變差，主要是因為網路頻寬的問題，故不建議部署超過1000個主節點．
    • hash_slot = CRC16(key) mod 16384
# 依次序set key value 操作Redis Server
set book1 abc
set book2 def
set book3 ghi
set book4 jkl
set book5 mno
set book6 pqr
set book7 stu

# Redis叢集資料Slot分流機制
                          client
                            |  (依上面次序進行set操作)
                            V
                  CRC16(key) mod 16384
--------------------------------------------------------------
       Redis               Redis               Redis
     (Master)            (Master)            (Master)
    |        |           |       |           |       |
    V        V           V       V           V       V
  Redis    Redis       Redis    Redis       Redis    Redis
(Replica) (Replica)  (Replica) (Replica)  (Replica) (Replica)
--------------------------------------------------------------
     book1=abc            book2=def             book3=ghi
     book4=jkl            book5=mno             book6=pqr
     book7=stu

Redis叢集架構實際配置

在配置Redis Server叢集架構時，基本上會配置三個Ｍaster節點，Master節點各自搭配一個以上的Replica節點，下面實際配置的內容會在單機進行，故會將Port分配來模擬叢集．

因需要在同一台機器模擬Redis Server叢集配置，故有以下前置作業需要先完成．

  • 叢集一
    • 6310-->Reids Server (Master) V
    • 6311-->Reids Server (Replica) V
    • 6312-->Reids Server (Replica) V
  • 叢集二
    • 6320-->Reids Server (Master) V
    • 6321-->Reids Server (Replica) V
    • 6322-->Reids Server (Replica) V
  • 叢集三
    • 6330-->Reids Server (Master) V
    • 6331-->Reids Server (Replica) V
    • 6332-->Reids Server (Replica) V
# 建立Redis叢集組態設定檔
# redis_cluster_6310.conf

# 伺服器組態檔
bind 127.0.0.1 -::1
daemonize yes
dir "/usr/local/redis/bin"
pidfile "/var/run/redis_6310.pid"
logfile "/tmp/redis_cluster_6310.log"
port 6310
protected-mode yes
# RDB+AOF持久化設定
save 3600 1 300 100 60 10000
dbfilename "dump_6310.rdb"
appendonly yes
appendfilename "appendonly_6310.aof"
aof-use-rdb-preamble yes
# 叢集設定
cluster-enabled yes
cluster-config-file cluster_nodes_6310.conf
cluster-node-timeout 15000

# redis_cluster_6311.conf
# 伺服器組態檔
bind 127.0.0.1 -::1
daemonize yes
dir "/usr/local/redis/bin"
pidfile "/var/run/redis_6311.pid"
logfile "/tmp/redis_cluster_6311.log"
port 6311
protected-mode yes
# RDB+AOF持久化設定
save 3600 1 300 100 60 10000
dbfilename "dump_6311.rdb"
appendonly yes
appendfilename "appendonly_6311.aof"
aof-use-rdb-preamble yes
# 叢集設定
cluster-enabled yes
cluster-config-file cluster_nodes_6311.conf
cluster-node-timeout 15000

# redis_cluster_6312.conf
# 伺服器組態檔
bind 127.0.0.1 -::1
daemonize yes
dir "/usr/local/redis/bin"
pidfile "/var/run/redis_6312.pid"
logfile "/tmp/redis_cluster_6312.log"
port 6312
protected-mode yes
# RDB+AOF持久化設定
save 3600 1 300 100 60 10000
dbfilename "dump_6312.rdb"
appendonly yes
appendfilename "appendonly_6312.aof"
aof-use-rdb-preamble yes
# 叢集設定
cluster-enabled yes
cluster-config-file cluster_nodes_6312.conf
cluster-node-timeout 15000

# redis_cluster_6320.conf
# 伺服器組態檔
bind 127.0.0.1 -::1
daemonize yes
dir "/usr/local/redis/bin"
pidfile "/var/run/redis_6320.pid"
logfile "/tmp/redis_cluster_6320.log"
port 6320
protected-mode yes
# RDB+AOF持久化設定
save 3600 1 300 100 60 10000
dbfilename "dump_6320.rdb"
appendonly yes
appendfilename "appendonly_6320.aof"
aof-use-rdb-preamble yes
# 叢集設定
cluster-enabled yes
cluster-config-file cluster_nodes_6320.conf
cluster-node-timeout 15000


# redis_cluster_6321.conf
# 伺服器組態檔
bind 127.0.0.1 -::1
daemonize yes
dir "/usr/local/redis/bin"
pidfile "/var/run/redis_6321.pid"
logfile "/tmp/redis_cluster_6321.log"
port 6321
protected-mode yes
# RDB+AOF持久化設定
save 3600 1 300 100 60 10000
dbfilename "dump_6321.rdb"
appendonly yes
appendfilename "appendonly_6321.aof"
aof-use-rdb-preamble yes
# 叢集設定
cluster-enabled yes
cluster-config-file cluster_nodes_6321.conf
cluster-node-timeout 15000

# redis_cluster_6322.conf
# 伺服器組態檔
bind 127.0.0.1 -::1
daemonize yes
dir "/usr/local/redis/bin"
pidfile "/var/run/redis_6322.pid"
logfile "/tmp/redis_cluster_6322.log"
port 6322
protected-mode yes
# RDB+AOF持久化設定
save 3600 1 300 100 60 10000
dbfilename "dump_6322.rdb"
appendonly yes
appendfilename "appendonly_6322.aof"
aof-use-rdb-preamble yes
# 叢集設定
cluster-enabled yes
cluster-config-file cluster_nodes_6322.conf
cluster-node-timeout 15000

# redis_cluster_6330.conf
# 伺服器組態檔
bind 127.0.0.1 -::1
daemonize yes
dir "/usr/local/redis/bin"
pidfile "/var/run/redis_6330.pid"
logfile "/tmp/redis_cluster_6330.log"
port 6330
protected-mode yes
# RDB+AOF持久化設定
save 3600 1 300 100 60 10000
dbfilename "dump_6330.rdb"
appendonly yes
appendfilename "appendonly_6330.aof"
aof-use-rdb-preamble yes
# 叢集設定
cluster-enabled yes
cluster-config-file cluster_nodes_6330.conf
cluster-node-timeout 15000

# redis_cluster_6331.conf
# 伺服器組態檔
bind 127.0.0.1 -::1
daemonize yes
dir "/usr/local/redis/bin"
pidfile "/var/run/redis_6331.pid"
logfile "/tmp/redis_cluster_6331.log"
port 6331
protected-mode yes
# RDB+AOF持久化設定
save 3600 1 300 100 60 10000
dbfilename "dump_6331.rdb"
appendonly yes
appendfilename "appendonly_6331.aof"
aof-use-rdb-preamble yes
# 叢集設定
cluster-enabled yes
cluster-config-file cluster_nodes_6331.conf
cluster-node-timeout 15000

# redis_cluster_6332.conf
# 伺服器組態檔
bind 127.0.0.1 -::1
daemonize yes
dir "/usr/local/redis/bin"
pidfile "/var/run/redis_6332.pid"
logfile "/tmp/redis_cluster_6332.log"
port 6332
protected-mode yes
# RDB+AOF持久化設定
save 3600 1 300 100 60 10000
dbfilename "dump_6332.rdb"
appendonly yes
appendfilename "appendonly_6332.aof"
aof-use-rdb-preamble yes
# 叢集設定
cluster-enabled yes
cluster-config-file cluster_nodes_6332.conf
cluster-node-timeout 15000

# 啟動三個sentinel
./redis-server redis_cluster_6310.conf 
./redis-server redis_cluster_6311.conf 
./redis-server redis_cluster_6312.conf 
./redis-server redis_cluster_6320.conf 
./redis-server redis_cluster_6321.conf 
./redis-server redis_cluster_6322.conf 
./redis-server redis_cluster_6330.conf 
./redis-server redis_cluster_6331.conf 
./redis-server redis_cluster_6332.conf 

# 確認是否正常運作
ps -ef|grep redis-server

https://ithelp.ithome.com.tw/upload/images/20211012/20111658xJoGVNoJ1l.png

# 目前每個Redis Server還是獨立的主節點
./redis-cli -h 127.0.0.1 -p 6310
127.0.0.1:6310> role
1) "master"
2) (integer) 0
3) (empty array)

./redis-cli -h 127.0.0.1 -p 6311
127.0.0.1:6311> role
1) "master"
2) (integer) 0

./redis-cli -h 127.0.0.1 -p 6312
127.0.0.1:6312> role
1) "master"
2) (integer) 0
  • 將三個Master加入叢集
./redis-cli -h 127.0.0.1 -p 6310

# 加入Redis Server Master 6320
127.0.0.1:6310> cluster meet 127.0.0.1 6320
OK

# 加入Redis Server Master 6330
127.0.0.1:6310> cluster meet 127.0.0.1 6330
OK

# 確認叢集Master節點狀況
127.0.0.1:6310> cluster nodes
1b9af54b8e0f609e0a066530b3f12a7852589fc3 127.0.0.1:6310@16310 myself,master - 0 1634049085000 1 connected
1307d911eb56f3079e2490dcab53b3131a0f8f81 127.0.0.1:6330@16330 master - 0 1634049087204 2 connected
7a0716a69a2e1af35eb353fadae030f6fc116864 127.0.0.1:6320@16320 master - 0 1634049086186 0 connected

# cluster_state:fail 的原因為資料分流的slot尚未分配完
127.0.0.1:6310> cluster info
cluster_state:fail
cluster_slots_assigned:0
cluster_slots_ok:0
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:3
cluster_size:0
cluster_current_epoch:2
cluster_my_epoch:1
cluster_stats_messages_ping_sent:87
cluster_stats_messages_pong_sent:92
cluster_stats_messages_meet_sent:2
cluster_stats_messages_sent:181
cluster_stats_messages_ping_received:92
cluster_stats_messages_pong_received:89
cluster_stats_messages_received:181
  • 分配資料分流的slot給各Master節點 (區間可自行分配)
  • 叢集一 0 ~ 5400
    • 6310-->Reids Server (Master) V
    • 6311-->Reids Server (Replica)
    • 6312-->Reids Server (Replica)
  • 叢集二 5401 ~ 10800
    • 6320-->Reids Server (Master) V
    • 6321-->Reids Server (Replica)
    • 6322-->Reids Server (Replica)
  • 叢集三 10801 ~ 16383
    • 6330-->Reids Server (Master) V
    • 6331-->Reids Server (Replica)
    • 6332-->Reids Server (Replica)
# 分配0 ~ 5400 slots 給6310-->Reids Server (Master)  
./redis-cli -h 127.0.0.1 -p 6310 cluster addslots {0..5400}
OK
# 分配5400 ~ 10800 slots 給6320-->Reids Server (Master)  
./redis-cli -h 127.0.0.1 -p 6320 cluster addslots {5401..10800}
OK
# 分配10801 ~ 16383 slots 給6330-->Reids Server (Master)  
./redis-cli -h 127.0.0.1 -p 6330 cluster addslots {10801..16383}
OK

# 再次確認叢集的狀態
./redis-cli -h 127.0.0.1 -p 6310

127.0.0.1:6310> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:3
cluster_size:3
cluster_current_epoch:2
cluster_my_epoch:1
cluster_stats_messages_ping_sent:711
cluster_stats_messages_pong_sent:699
cluster_stats_messages_meet_sent:2
cluster_stats_messages_sent:1412
cluster_stats_messages_ping_received:699
cluster_stats_messages_pong_received:713
cluster_stats_messages_received:1412

# 確認slots分配是否如預期
127.0.0.1:6310> cluster slots
1) 1) (integer) 0
   2) (integer) 5400
   3) 1) "127.0.0.1"
      2) (integer) 6310
      3) "1b9af54b8e0f609e0a066530b3f12a7852589fc3"
2) 1) (integer) 5401
   2) (integer) 10800
   3) 1) "127.0.0.1"
      2) (integer) 6320
      3) "7a0716a69a2e1af35eb353fadae030f6fc116864"
3) 1) (integer) 10801
   2) (integer) 16383
   3) 1) "127.0.0.1"
      2) (integer) 6330
      3) "1307d911eb56f3079e2490dcab53b3131a0f8f81"
  • 將Master加入Replica增加高可用機制
  • 叢集一 0 ~ 5400
    • 6310-->Reids Server (Master)
    • 6311-->Reids Server (Replica) V
    • 6312-->Reids Server (Replica) V
  • 叢集二 5401 ~ 10800
    • 6320-->Reids Server (Master)
    • 6321-->Reids Server (Replica) V
    • 6322-->Reids Server (Replica) V
  • 叢集三 10801 ~ 16383
    • 6330-->Reids Server (Master)
    • 6331-->Reids Server (Replica) V
    • 6332-->Reids Server (Replica) V
# 取的個主節點的master node id
./redis-cli -h 127.0.0.1 -p 6310

127.0.0.1:6310> cluster nodes
1b9af54b8e0f609e0a066530b3f12a7852589fc3 127.0.0.1:6310@16310 myself,master - 0 1634050264000 1 connected 0-5400
1307d911eb56f3079e2490dcab53b3131a0f8f81 127.0.0.1:6330@16330 master - 0 1634050265828 2 connected 10801-16383
7a0716a69a2e1af35eb353fadae030f6fc116864 127.0.0.1:6320@16320 master - 0 1634050266837 0 connected 5401-10800

# 6310-->Reids Server (Master) 
# master node id => 1b9af54b8e0f609e0a066530b3f12a7852589fc3

# 將Reids Server 6311 6312 加入叢集
./redis-cli -h 127.0.0.1 -p 6310
127.0.0.1:6310> cluster meet 127.0.0.1 6311
OK
127.0.0.1:6310> cluster meet 127.0.0.1 6312
OK

# 將Reids Server 6311 成為 6310 的 Replica
./redis-cli -h 127.0.0.1 -p 6311
127.0.0.1:6311> cluster replicate 1b9af54b8e0f609e0a066530b3f12a7852589fc3
OK
127.0.0.1:6311> exit

# 將Reids Server 6312 成為 6310 的 Replica
./redis-cli -h 127.0.0.1 -p 6312
127.0.0.1:6312> cluster replicate 1b9af54b8e0f609e0a066530b3f12a7852589fc3
OK
127.0.0.1:6312> exit

# 6320-->Reids Server (Master) 
# master node id => 7a0716a69a2e1af35eb353fadae030f6fc116864

# 將Reids Server 6321 6322 加入叢集
./redis-cli -h 127.0.0.1 -p 6320
127.0.0.1:6320> cluster meet 127.0.0.1 6321
OK
127.0.0.1:6320> cluster meet 127.0.0.1 6322
OK

# 將Reids Server 6321 成為 6320 的 Replica
./redis-cli -h 127.0.0.1 -p 6321
127.0.0.1:6321> cluster replicate 7a0716a69a2e1af35eb353fadae030f6fc116864
OK
127.0.0.1:6321> exit

# 將Reids Server 6322 成為 6320 的 Replica
./redis-cli -h 127.0.0.1 -p 6322
127.0.0.1:6322> cluster replicate 7a0716a69a2e1af35eb353fadae030f6fc116864
OK
127.0.0.1:6322> exit

# 6330-->Reids Server (Master) 
# master node id => 1307d911eb56f3079e2490dcab53b3131a0f8f81

# 將Reids Server 6323 6332 加入叢集
./redis-cli -h 127.0.0.1 -p 6330
127.0.0.1:6330> cluster meet 127.0.0.1 6331
OK
127.0.0.1:6330> cluster meet 127.0.0.1 6332
OK

# 將Reids Server 6331 成為 6330 的 Replica
./redis-cli -h 127.0.0.1 -p 6331
127.0.0.1:6331> cluster replicate 1307d911eb56f3079e2490dcab53b3131a0f8f81
OK
127.0.0.1:6331> exit

# 將Reids Server 6332 成為 6330 的 Replica
./redis-cli -h 127.0.0.1 -p 6332
127.0.0.1:6332> cluster replicate 1307d911eb56f3079e2490dcab53b3131a0f8f81
OK
127.0.0.1:6332> exit
  • 確認Redis叢集配置
./redis-cli -h 127.0.0.1 -p 6310
127.0.0.1:6310> cluster nodes
d20a8414e90365895a2432ad98d35b42d02228c1 127.0.0.1:6322@16322 slave 7a0716a69a2e1af35eb353fadae030f6fc116864 0 1634051854937 3 connected
d27292294e3880171277b21e1488b454e9d0d716 127.0.0.1:6332@16332 slave 1307d911eb56f3079e2490dcab53b3131a0f8f81 0 1634051854000 2 connected
1307d911eb56f3079e2490dcab53b3131a0f8f81 127.0.0.1:6330@16330 master - 0 1634051853000 2 connected 10801-16383
48b1b701f528817c932ef7cc17b899c428284d88 127.0.0.1:6321@16321 slave 1307d911eb56f3079e2490dcab53b3131a0f8f81 0 1634051852000 2 connected
7a0716a69a2e1af35eb353fadae030f6fc116864 127.0.0.1:6320@16320 master - 0 1634051854000 3 connected 5401-10800
82013de2c546919a43c9b6597a2b6035bb1ac930 127.0.0.1:6312@16312 slave 1b9af54b8e0f609e0a066530b3f12a7852589fc3 0 1634051854000 1 connected
e3b71c1c0414ea6ec90b5d37a0ff2c963e2a93ec 127.0.0.1:6331@16331 slave 1307d911eb56f3079e2490dcab53b3131a0f8f81 0 1634051857991 2 connected
eca9f94125a621846fe9c5e92d60003539f0a512 127.0.0.1:6311@16311 slave 1b9af54b8e0f609e0a066530b3f12a7852589fc3 0 1634051856973 1 connected
1b9af54b8e0f609e0a066530b3f12a7852589fc3 127.0.0.1:6310@16310 myself,master - 0 1634051853000 1 connected 0-5400

上一篇
Day26 Redis架構實戰-Redis叢集架構/Gossip協議
下一篇
Day28 Redis架構實戰-Redis叢集指令
系列文
Redis還在學 共 30 篇
  1. 26
    Day26 Redis架構實戰-Redis叢集架構/Gossip協議
  2. 27
    Day27 Redis架構實戰-Redis叢集Slot分流機制
  3. 28
    Day28 Redis架構實戰-Redis叢集指令
  4. 29
    Day29 Redis架構實戰-Redis-cli指令
  5. 30
    Day30 Redis架構實戰-Redis Request Routing/效能監控與調教
完整目錄
圖片
  直播研討會
圖片
{{ item.channelVendor }} {{ item.webinarstarted }} |
{{ formatDate(item.duration) }}
直播中

尚未有邦友留言

立即登入留言
iThome鐵人賽
參賽組數
1064 組
團體組數
40 組
累計文章數
22137 篇
完賽人數
596 人
iT+看影片追技術 看影片追技術 看更多
熱門tag 看更多
15th鐵人賽 16th鐵人賽 13th鐵人賽 14th鐵人賽 12th鐵人賽 11th鐵人賽 鐵人賽 2019鐵人賽 javascript 2018鐵人賽 python 2017鐵人賽 windows php c# windows server linux css react vue.js
熱門問題
熱門回答
熱門文章
IT邦幫忙